Learning Visibility for Robust Dense Human Body Estimation

AbstractEstimating 3D human pose and shape from 2D images is a crucial yet challenging task. While prior methods with model-based representations can perform reasonably well on whole-body images, they often fail when parts of the body are occluded or outside frame. Moreover, these results usually do not faithfully capture silhouettes due to their limited representation power deformable models (e.g., representing only naked body). An alternative approach estimate dense vertices predefined template in image space. Such effective localizing within an but cannot handle out-of-frame parts. In this work, we learn estimation that robust partial observations. We explicitly model visibility joints x, y, z axes separately. The x y help distinguishing cases, depth axis corresponds occlusions (either self-occlusions by other objects). obtain pseudo ground-truths labels UV correspondences train neural network predict along coordinates. show serve as 1) additional signal resolve ordering ambiguities self-occluded 2) regularization term fitting predictions. Extensive experiments multiple datasets demonstrate modeling significantly improves accuracy estimation, especially for partial-body cases. Our project page code at: https://github.com/chhankyao/visdb.